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ABSTRACT 

Assume  F  and  G  are  distributions  on  [0,°°)  with 
densities  f  and  g  ,  respectively.  If  G  ^F  is  convex  on 
the  support  of  F  (an  interval),  then 


r(x)  -  G_1F(x) 


M 


g[G_1F (x) ] 


(the  generalized  failure  rate  function)  is  nondecreasing  in 
x  e  .  We  assume  G  known,  r  nondecreasing  and 

consider  the  problem  of  estimating  r  .  A  general  class  of 
isotonized  fixed  and  random  "window"  estimators  are  proposed 
and  studied.  These  include  the  maximum  likelihood  estimators 
(HLE's)  studied  by  Grenander  (1956),  Marshall  and  Proschan 
(1965),  and  Prakasa  Rao  (1966).  By  appropriate  choice  of 
the  window  size,  we  improve,  asymptotically,  on  the  MLE. 
Strong  consistency  is  proved  for  isotonic  window  estimators 
generalizing  and  simplifying  previous  proofs.  Strong 
consistency  is  proved  for  a  class  of  isotonic  estimators 
when  the  basic  estimator  is  strongly  consistent. 
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PART  I:  STRONG  CONSISTENCY 
by 

Richard  E.  Barlow 
Willem  R.  van  Zwet 

1.  ISOTONIC  ESTIMATION  IN  THE  CASE  OF  CONVEX  ORDERING 

Assume  F  and  G  are  distributions  on  [0,®)  with  densities  f  and  g  , 
respectively.  We  consider  the  following  convex  partial  ordering  (c-ordering)  on 
the  space  of  distribution  functions  on  [0,°°)  .  We  say  that  F  is  c-ordered  with 
respect  to  G  [F  <  G]  if  and  only  if  G  ^F  is  convex  on  the  support  of  F  (an 
interval)  [van  Zwet  (1964 )  ] .  We  say  that  F  =  G  if  G  '*"F(x)  ■  ax  +  b  for  some 
a  >  0  . 

If  F  <  G  ,  then 
c 


(1.1) 


r(x)  -  ^  G  1F(x) 


1M. 


g[G_1F(x)] 


is  nondecreasing  in  x  e  [0,®)  .  We  assume  G  known,  F  <  G  and  consider  the 
problem  of  estimating  r  .  Maximum  likelihood  estimators  (MLE)  for  r  in  the  case 
when  G  is  the  exponential  or  the  uniform  distribution  have  been  investigated  by 
Grenander  (1956),  Marshall  and  Proschan  (1965),  B.L.S.P.  Rao  (1966),  and 
Robertson  (1967). 

Let  r  be  an  initial  or  basic  estimator  for  r  .  Let 


n 

0  =  w„  <  w,  <  ...  <  w.  <  ...  be  a  subdivision  of  [0,®)  and  u  {w.  }  a 

0,n  l,n  i,n  n  j,n 

sequence  of  weights  on  Jw.  )  .  Assume  w,  <  x  <  w,  .  .  We  call  the  inter- 

t  J,nfj-o  i,n  ~  i+1»n 

val  [w.  ,w, . ,  ]  a  "window."  We  will  be  especially  interested  in  the  effect  on 

l,n  1+1, n 

the  estimators  with  respect  to  the  rapidity  with  which  the  window  goes  to  zero. 
Following  Brunk,  we  call 


2 


(1.2) 


r„(x) 


inf  sup 
t>i+l  s<i 


-  j- 

y  r  (w  )u  (w.  } 

l=s  n  J-n  n  J»n 

t-l 

l  U  (w.  } 

4  n  J  ,  n 
j  =  s 


the  monotonic  regression  or  more  generally  the  isotonic  regression  of  r^  with 

respect  to  the  discrete  measure  •  Note  that  r^  is  a  nondecreasing  step 

function.  Let  be  the  empirical  distribution  function  corresponding  to  an 

ordered  sample  0  <  X,  <  X„  <  . . .  <  X  from  F  .  Let  »  (w..,  -  w,  )/2  . 

u  X  2.  n  i  li  l  ,n  1 1  n 

We  call 


r  (5  ,  .  W  .  Fn(Vl.n>  -  VVn1 

r"  1  gc-^u,)  -  >'l>n)*G‘1rn(51> 


the  "naive"  estimator  for  r(£  J  •  This  has  been  extensively  studied  by 

Parzen  (1962)  and  others  (in  the  case  G(x)  =  x  (0  <_  x  <_  1)  )  and  by  Watson  and 

Leadbetter  (1964)  and  others  (in  the  case  G(x)  *  1  -  e  X  for  x  >_  0  )  . 

If  r  is  the  naive  estimator  and  u  {£,}  *•  gG  ^F  (C,)(w,,1  -  w.  )  ,  then 

n  n  i  n  i  i+i ,  n  I ,  n 


(1.3) 


„  F  (w^  )  -  F  (w  ) 

r  (x)  *  inf  sup  — - — — - ^ - 

t— i+l  s— V  gG~^F  (C  )(w  -  w  ) 

jis  8  V\jMWj+l.n  j,n; 


If  the  grid  {w,  }  is  fixed  and  not  random,  then  (1.3)  is  the  MLE  (in  the  case 

i ,  n 

G(x)  ■  1  -  e  X)  for  a  discrete  IFR  distribution.  [Cf.  Marshall  and  Proschan  (1965), 


Section  7.]  If  w.  =  X,  ,  then 
i,n  l 


(1.4) 


r  (x)  -  inf  sup  — — - - - - - - 

t>i+l  s<i  r  -1_  . 

-  n^gG  Fn(C;|)(Xj+1  -  Xj) 


is  the  MLE  studied  by  Grenander  (1956),  Marshall  and  Proschan  (1965),  and 


I 
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B.L.S.P.  Rao  (1966)  (in  the  case  G(x)  -  1  -  e  X  for  x  >_  0 )  and  by 
Robertson  (1967)  (in  the  case  G(x)  *  x  ,  0  <_  x  _<_  1)  .  If  G(x)  *  1  -  e  x  ,  then 
the  weight  ■  fl/nKn  -  i)  (X^+^  -  X^)  is  the  "total  time  on  test"  (divided  by 

n)  between  ordered  observations  X^  and  X^+^  .  We  show  in  Section  2  that  in  this 
case 


<1.5' 


■  -.'V  •  / 


F~1(i/n) 


gG  (u)du 
n 


-  (n)  a'"*’3'  ♦p(y)  “  J  8G  1F(u)du 


as  i/n  ■+■  y  and  n  -*  00  .  We  call  4>p(y)  the  total  time  on  test  distribution  or 

measure  (not  necessarily  a  probability  distribution). 

If  r  is  the  naive  estimator  and  y  {£.}  *  w , , .  -  w.  ,  then 

n  n  i  i+l,n  i,n 


(1.6) 


* 

r  (x) 


inf 

t>i+l 


sup 

s<i 


t-1  [F  (w  )  -  F  (w  )] 
_ £  n  -1xl»n _  n  lin— 


w^  -  w  j“S 
t,n  s,n 


is  the  lsotonized  "naive"  estimator.  Note  that  rn(x)  is  a  special  kind  of  average 

with  respect  to  "discrete  Lebesgue  measure,"  while  r^(x)  is  a  special  kind  of 

average  with  respect  to  "totai  time  on  test"  weights.  It  might  be  conjectured  that 

r  (x)  will  perform  better  than  r  (x)  for  small  samples  since  the  "total  time  on 
n  □ 

test"  for  an  interval  is  a  measure  of  our  information  over  that  interval  and 

r  (x)  takes  advantage  of  that  fact.  However,  as  we  shall  show  in  the  companion 
n 

paper^,  they  are  asymptotically  equivalent  (when  the  windows  of  the  grid  are  not 
too  wide) . 

Another  estimator  for  r  can  be  obtained  using  the  "graphical"  estimator 
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(1.7) 


G'1*"  (w  )  -  G_1F  (w.  ) 

n  1+1 ,  n _ n  i,n 

w.,.  -  w, 

i+l,n  i.n 


This  is  motivated  by  the  identity 


0'lF(wi+i.»)  -  g'1f(ui 


i+l,n 


)du  . 


If  we  use  this  for  our  basic  estimator  and  let  y  {£,}**  w.,,  -  w.  ,  we  obtain 

n  i  i+l,n  i,n 


(1.8) 


r  (x)  -  inf  sup 
n  t>i+l  s<i 


G_1F  (w  )  -  G_1F  (w  ) 
_ n  t.n  n  s.n 

-  w 

t.n  s.n 


Although  all  three  estimators,  r  (x)  ,  r  (x)  ,  r  (x)  will  be  shown  to  be 

n  n  n 

it 

asymptotically  equivalent  (for  certain  grids),  we  can  see  that  rn(x)  an^  rn^x^ 
are  perhaps  more  alike  since  they  use  the  same  weighting  function.  The  similarity 

is  also  apparent  if  we  expand  G  ^  in  a  Taylor's  series  about  ^Q(x)  •  Notice 

-  * 

that  when  G(x)  »  x  (0  <  x  <  1)  ,  r  (x)  ■  r  (x)  -  r  (x)  . 

—  —  n  n  n 
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2.  CONVERGENCE  OF  TOTAL  TIME  ON  TEST  MEASURES 

The  total  time  on  test  measure,  $  ,  plays  a  crucial  role  in  proving  strong 

r 

n 

A 

consistency  of  rQ(x)  •  It  also  is  important  in  life  test  theory  for  the 
exponential  distribution.  Note  that  when  G(x)  ■  1  -  e  for  x  >_  0  , 

F^U/n) 

gG-1Fn(u)du 

'  jx  8G'1f»(xj-i)(xj  -  XJ-1> 

-i  ("-J+lHXj  -  Xj.i)  . 

This  transform  was  introduced  in  Marshall  and  Proschan  (1965)  and  exploited  by 
B.L.S.P.  Ra«^  (1966)  (when  G(x)  ■  1  -  e  X)  to  obtain  the  maximum  likelihood 
estimate  for  the  failure  rate  function  of  monotone  failure  rate  distributions. 

GO 

Assume  -  f  xdF(x)  <  00  .  Strong  uniform  convergence  of  4^  (y)  to  't’p(y)  , 

0  n 

in  this  case,  is  an  easy  consequence  of  the  Glivenko-Cantelli  theorem  and  the 
strong  law  since 


♦  (1)  -  A  l  (n-J-L)  (X  -  I  )  -  A  l  X  u  . 

o  ”  j-1  ]  1  1  ”  j-1  1  1 

However,  strong  uniform  convergence  is  not  trivial  for  general  distributions,  G  , 
when  F  *(1)  ■  00  . 

The  proof  of  the  following  theorem  is  due  to  H.  D.  Brunk.  Note  that  in  the 
following  theorem  we  do  not  assume  F  <  G  . 

Theorem  2.1: 


Suppose  that  either 


(2.1) 


F_1 (1)  <  » 


(2.?) 


as 

_1(1)  ■  "  »  j 


xdF(x)  <  °° 


oo 

.  /  gs'1 


F(x)dx  <  °° 


(2.3) 


has  a  continuous  derivative  ip  on  [0,1]  .  Then 


[sup  | * 
0<y<l  n 


(y)  -  *F(y)|  o 


^Note  that  F  <  G  implies  J"  gG  ^F(x)dx  <  00  .j 

To  prove  Theorem  2.1,  we  need  the  following  lemma: 


Lemma  2.2: 


Let  ijj  :  [0,1]  R  be  continuous  and  set  f( 
For  positive  integers  i  ,  n  ,  i  <_  n  ,  set 


-I 


^ (t)dt  ,  u  e  [0,1]  . 


V  ■  iHr)  -  ii) 


Let  Z^,Z2»  . . . ,  be  independent  random  variables,  each  with  continuous 
distribution  F  .  Let  |ZjJ  and  | Z^<|»[F(Z^)]  |  each  have  finite  mean  For  fixed 
positive  integer  n  ,  let  . .  . ,  be  the  order  statistics  of  Z^,  ...,  Z^ 


n  r 

lio  l  A  1X1  ai8,  -  I  x^/[F(x)]dF(x)  . 
n -*•  i-1  *  \ 
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Proof : 


We  show  first  that 


0  . 


Set  ■  F(X^)  .  Then  U^,  are  order  statistics  of  a  random  sample  from 

the  uniform  distribution  over  [0,1]  .  We  have 

i/n 

-A  .  -  7  *[F(X.)]  -  f  [ip(u)  -  iKU  )]du  . 

n,  i  n  i  J  l 

(i-l)/n 

By  the  Glivenko-Cantelli  theorem, 

lim  sup  sup  | u  —  U  |  a»s’  o  . 

n  l<i<n  ue[ (i-l)/n,i/n] 


But  ij>  is  uniformly  continuous  on  [0,1]  .  Hence  with  probability  1,  given  c  >  0  , 
there  exists  such  that  n  >_  implies  |^(u)  -  |  <  £  for  all 

u  e  ,  i  ■  1,2,  n  where  u  denotes  a  sample  point.  Then 

n  _>  Ne(oi)  implies 


C.  (a  , 

i\  n,  i 


1'[F(X1)]) 


But  with  probability  1, 


1 

n 


E | ZL |  ,  hence  with  probability  1 


lim  sup 
n 


n,i 


«l»[F(Xi)]) 


9 


and 


lta  j,  |Xi(Vi  +  n  ♦,F<V1)|  V'  0  • 

n  I“1  1  1 


On  the  other  hand,  it  follows  from  the  strong  lfv  that 


-  I  X.i|i[F(X .)]  f  X{p[F(x)  ]dF(x)  . 

n  i-1  1  1  J 

1  0 


The  conclusion  of  the  lemma  follows.  | | 


Proof  of  Theorem  2.1: 


Let  0  =  Xq  <_  <_  X2  ±  ±  be  an  ordered  sample  from  I 


J«)  ■  j,  ^(Hr)  (xi  -  xj-i> 


V" 


Case  1: 


Suppose  that  (2.1)  holds.  Then  for  y  e  [0,1]  we  have 


I (y)  -  *F(y)! 

n 


- 


F^ty) 


'/ 


F'^y) 


[4* (F^(x))  -  Y(F(x))]dx 
n 


F_1(y) 


¥(F(x))dx  -  J  Y(F(x))dx 
0 


The  first  term  on  the  right  converges  to  0  with  probability  1  by 
Cantelli  theorem,  and  the  second  converges  to  0  with  probability 
law. 

Case  2: 


Then  F_1(l)  -  ®  , 


so  that 


the  Glivenko- 
1  by  the  strong 


Suppose  that  (2.2)  and  (2.3)  hold. 


which  implies 


9 


*(  1) 


g (G~ 1 (1)  ] 


0  .  For  y  <  1  ,  the  argument  of  Case  1  is  valid. 


Integrating  by  parts,  we  have 


F_1(y) 


>p(y)  "  f  gG-1F(x)dx  -  y*  gG-1(u)dF-1 

0  0 


(u) 


7 

f 


4'(u)dF~1(u)  -  4'(y)F 


1<y>  -  /  F"1 


(uH(u)du  . 


F^Cy) 

*F(y)  -  V (y) F_1  (y)  -  J  xip[F(x)]dF(x)  . 

0 


Since  by  (2.2),  J  gG  ^F(x)dx 
0 


®  ,  we  have 


*p(l)  ■  -  j  xtj/[F(x)]dF(x)  <  °°  . 


Set  g.  •  gG  1(k/n)  ,  k  -  0,1,  n 


Ip(u)  -  V  (u)  ,  U  E  [0,1]  , 


and 


Then 


*n,iXi  +  «jXj 


j  -  1,2, 


n 


and 


10 


(I) 


l 


1-1 


A  .X. 
n,i  ■ 


since  gG  ^(1)  -  0  .  The  conclusion  '!■  (1)  a‘-*s’  $  (1)  now  follows  from 

r  r 

n 

Lemma  2.2.  j  ] 

In  applying  Theorem  2.1  to  prove  consistency  in  Section  3,  we  assume  F  =  G 
so  that  Conditions  (2.2)  and  (2.3)  become 


(i)  J  xdG(x)  <  ®  and 
0 

(ii)  gG  ^  has  a  continuous  derivative  on  [0,1]  . 

An  alternative  condition  on  G  is  3  a  number  0  <  n  <  1  9  for  y  n  ,  gG  *(y) 

2G  ^  (y ) 

is  nonincreasing  and  °  is  nondecreasing.  To  prove  that  these  conditions 

are  sufficient,  we  state  some  additional  results. 


Lemma  2.3: 

Let  Z^  n  ,  i  -  1,2,  ...,  an  ,  n  -  1,2,  ...  have  exponential  distributions 

with  mean  1  and  assume  that  for  every  fixed  n  ,  Z,  ,Z_  ,  ....  Z  are 

l.n  Z,n  a 

an  n,n 

independent.  Then,  if  Yog"n  *  °°  ^or  n  00  » 


lim  — 
a 

n-*»  n 


a 

n 

l 

i-1 


1 


with  probability  one. 

The  proof  is  a  straightforward  application  of  the  Borel-Cantelli  lemma,  and  we 
omit  it. 

Lemma  2.4: 

Let  X^,X2»  ...  be  independent  and  identically  distributed  with  continuous 
distribution  function  F  and  let  denote  the  empirical  distribution  function 


11 


based  on  X. .....  X  .  Then 

1’  ’  n 


11m 

n-**> 


log  n 


sup  |Fn(x) 
x 


F  (x) 


0 


with  probability  1. 


Proof; 

From  the  exact  distribution  of  the  one-sided  Kolmogorov-Smirnov  test  statistic, 
we  infer 


P(sup  )Fn(x)  -  F(x)|  >  z)  <_  2P(sup  (Fn(x)  -  F(x))  >  z) 


[ n  (1— z )  ]  .  .  vn-i,  4  \1-1 

22  l  (;)(*- *-*)  (-;) 


0  <  z  <  1  . 


Consider  the  function 


f(z)  -  log  (")  +  (n  -  i)  log  (l  -  z  -  -^)  +  i  log  (z  +  “)  +  2nz2  , 


for  0  <_  i  <_  n(l  -  z)  ,  i.e.,  for  z  <_  1  -  i/n  ,  i  •  0,1,  ...,  n  .  For  z  =  0  , 
the  value  of  f  is  the  logarithm  of  a  binomial  probability,  hence  f  (0)  <_  0  . 
Furthermore,  for  0  z  1  -  i/n  , 


f  ’  (z) 


> 


and  as  a  result  f(z)  <  0  for  all  z 


under  consideration.  Hence 


and 


i 
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P (sup  | F  (x)  -  F(x)  |  >  z)  <  2e'2nZ  [  — ~  <_  2ne"2nz  . 

i-0  z  +  - 
n 

It  follows  that  for  every  e  >  0 

P(l^^  SUP  lFn<x>  "  F(x>l  >  E)  1  2  I  nl~2£  108  n  <  00  > 

and  the  lemma  is  proved  by  applying  the  Borel-Cantelli  lemma. 

Lemma  2.5: 

Let  ^2.’^2’  *'*  independent  and  identically  distributed  according  to  an 

exponential  distribution  with  mean  1  and  let  (0  ■  <) 

0:n 

Y.  <  Y„  <  ...  <  Y  denote  the  order  statistics  corresponding  to 
l:n  2:n  n:n  v  6 

Y^.Y^,  •  Y^  .  Then  for  any  0  <_  c  <  1  , 


lim 
a*®  [cn]+l 


l  (Y.  -  Y,  )  max  (l  -  ‘3-r—  ,  e  J_1:n)  -  1  - 

V. ,  j  :n  j-1 :n  \  n  ’  / 


with  probability  1. 


Proof ; 


Define  ts 


(n-j+1) (Y 


-  Y. 


.  i  ...  .  /  .  Since  2.  *  Z .  .  . . . .  Z 

j,n  J  j:n  j-l:n  l,n*  2,n*  *  n,n 

independent  exponential  distributions  with  mean  1  , 


have 


l 

[cn]+l 


j-l:n  \  n  / 


1 

n 


l 

[cn]+l 


converges  almost  surely  to  (1  -  c)  for  n  -*•  «  by  Lemma  2.3.  Define 
'Y1:n 

U.  ■  1  -  e  J  .  Then  (0  ■  <)  U.  <  ...  <  U  are  distributed  as  order 

J  :n  0:n  l:n  n:n 

statistics  from  a  uniform  distribution  on  (0,1)  .  To  prove  the  lemma,  it  suffices 


to  show  that 
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2/3  1/3 

converges  to  zero  a.s.  Let  ■  n  -  n  and  b^  *  n  -  n  ,  then 

an  '  1  -1/3  .  .  bn  '  1  -2/3 

1 - -  n  and  1  -  -  >  n  ,  hence 

n  —  n  — 


1 

n 


b 

n 


l 


1-a  +1 
J  n 


1/3 

sup  n 

a  +l<j<b 
n  — n 


U 


J-l:n 


l 


j-b  +1 
J  n 


By  Lemmata  2.3  and  2.4,  the  right-hand  side  of  this  inequality  tends  to  zero  almost 
surely,  which  completes  the  proof.  |  | 

Theorem  2.6: 

Let  G  be  a  continuous  distribution  function  on  [0,°°)  with  density  g  and 

assume  that  there  exists  a  number  0  <  n  <  1  such  that  for  y  >_  n  ,  gG  (y)  is 

-1 

nonincreasing  and  - is  nondecreasing.  Then  for  every  y  <  c  <  1  , 

(*G 

n-*»  \  n 


with  probability  1.  Hence, 


11m 

n-*» 


V1’  -  ‘g'1') 


0 


with  probability  1. 
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Proof : 


*G  <»  -  »G  ‘  .  I  (XJ:„  - 

n  n  [cn]+l 


where  X  is  an  order  statistic  from  G  .  If  we  define  Y,  by 
j:n  J:n 

G~  Ml  -  e  -  X.  ,  then  YJ  is  an  order  statistic  from  an  exponential 

'  J  :n  J  :n 

distribution  with  mean  1  .  With  probability  1,  there  exists  a  number  N  such 

'V1:n  >  „  .  Sinc(i  afilui 


that  for  n  >_  N  and  J  >_  [cn]  +1  we  have  1  - 


1  -  y 


is  nondecreasing  for  y  >_  n  ,  we  have  for  n  ^  N  and  j  >.  [cn]  +  1 


* 

~YJ  :n 
e  J 


-Y 


j-l:n 


—  ^Yj:n  Yj-l:n^  .(  -Y  \ 
gG'Ml  -  e 

j-l:n  „  J 


gG 

-Y, 


where  Y,  ,  <  Y  <  Y  .  If  1  -  e 

j-l:n  -  j  :n  —  j  :n 

j  _>  [cn]  +  1 


—  n 


,  we  have  for  n  >_  N  and 


<  (Y,  -  Y.  .  )fl  -  ^  ~  M 

—  j :n  j-l:n  V  n  / 


gG~1(‘^  ~~  ~ )  (X,  -  X.  .  )  < 

6  \  n  /  j:n  j-l:n  - 

'Y1-l:n  .-1/1  -  1\ 

e  -8° 


(l  -  J-j-i)  •  gG~ 1  ( 1  -  e" 

-1  -Y  ^ 

again  since  ^  nonc*ecreasing  f°r  Yin.  If  1  -  «  ^  ‘n<^ 

we  use  the  fact  that  gG  My)  is  nonincreasing  for  y  _>  n  to  obtain 


gG 


~-1)  (X.  -  X,  .  )  <  (Y  -  Y  )e  •J‘1:n  . 

V  n  /  j:n  j-l:n  —  j:n  j-l:n 
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Hence  for  n  >  N  , 


*0  (1)  -  *0  (Jf1)  -  ,  L 

n  n  [cnj+l 


^J-l:n)  max  l1 


Application  of  Lemma  2.5  completes  the  proof.  || 

We  will  be  mainly  interested  in  isotonic  estimators  based  on  grids  with  wider 
"windows"  than  those  provided  by  order  statistics.  We,  therefore,  define  an 


analogue  to  the  empirical  distribution  for  more  general  grids.  Let 
a  subdivision  of  [0,°°)  and  define 


NL 


Fn,N^  "  Fn(wi,lP 


for  w  <  x  <  w 

i,N  -  1+1, N 


00 

w  )  becomes  dense  in  [0,®)  as  N  -*•  °°  in  the  sense  that  given 

*  fi-1 

_  *  * 

e  >  0  ,  3  N  3  for  arbitrary  x  and  all  N>N  3  n  5  ^  t  (x  -  e  ,  x  +  e) 

* 

Clearly,  F  „(x)  <  F  (x)  for  all  x  and 
1  n,N  —  n 


11m  F  (x)  -  F  (x) 
N-h*>  n’N  n 


since  F  „(x)  *  F  (x)  when  the  grid  includes  the  order  statistics.  Also, 

n,N  n 

F  (x)  -  F  (x)  ■*  F(x)  uniformly  in  x  e  [0,b]  for  every  b  <  °°  .  The  proof 

Q|Q  0  & •  S  • 

is  similar  to  the  proof  of  the  Glivenko-Cantelli  theorem. 


Theorem  2.7: 

Suppose  the  grid  iw  l  becomes  dense  in  [0,°°)  in  such  a  way  that 
l  1,N/i-l 

N'  >  N  =>iw.  \°°  c  fw  N,)  .  Suppose  that  either  Condition  (2.1)  or  (2.2) 

l  i*  li-1  (  ’  fi-1 

and  (2.3)  of  Theorem  2.1  hold.  Then 


[sup  | *  * 
Q^l  Fn 


(y)  -  ♦p(y)  I  o 


i”J  =•  l  . 
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Proof : 


The  proof  is  easy  if  (2.1)  holds.  Hence,  suppose  (2.2)  and  (2.3)  hold.  Since 

ao 

/-l  *  * 

gG  F(x)dx  <  00  ,  given  e  >  0  3  N  9  for  N  >  N 

0 


00 


l 

i-0 


gG  F^Wi,N^Wi+l 


-  -  $p(l) 


< 


£ 

2  ‘ 


Consider  the  discretization  of  F  ;  i.e., 

Fd>N(x)  -  F(w±)  for  w±  <  x  <  w1+1 


and  apply  Theorem  2.1  to  F^  ^  .  Hence, 


P 


nE(N,uj)  9 


I*  *  (1)  -  *  (1)|  <  e 

F  b 

n,N 


and 


I  i>  (1)  -  <t>  (1)  |  <  e  for  all  n  >  n  (N,io)  and  all 
F  F  E 

n 


i 


where  w  denotes  a  sample  point.  Since 

follows  that  <t>  A  (1)  is  closer  to  4> 

F  n 

n,N 

convergence  is  uniform  in  both  n  and  N 


N'  >  N  implies  {wi  C  {w^  }  ,  it 

than  $  .  is.  It  follows  that  the 

f„.n 

so  that  <5>  .  (1)  3>  (1)  •  1  I 

p  a.s.  t 

n 
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3.  STRONG  CONSISTENCY  OF  ISOTONIC  ESTIMATORS  BASED  ON  TOTAL  TIME  ON  TEST  MEASURE 

A 

We  assume  F  <  G  and  show  that  r  (x)  -+■  r(x)  with  probability  one  at 
c  n 

continuity  points  of  r  .  We  use  the  fact  that,  under  regularity  conditions,  the 

total  time  on  test  distribution  $  is  uniformly  strongly  consistent.  This 

F 

n 

generalizes  and  simplifies  consistency  proofs  of  Marshall  and  Proschan  (1965)  for 
the  case  G(x)  ■  1  -  e  and  Robertson  (1967)  for  the  case  G(x)  *  x  (0  x  £  1)  . 
The  first  part  of  the  proof  is  similar  to  that  of  Marshall  and  Proschan  (1965) 
while  the  use  of  the  uniform  consistency  of  4>  was  suggested  by  the  proof  of 

f 

n 

Robertson  (1967). 

Theorem  3.1; 

Suppose  that 

(i)  r(x)  -  f(x)/gG  *F(x)  is  nondecreasing  in  x  _>  0  ; 

(ii)  either  (w  }  is  the  grid  determined  by  the  order  statistics  or 

i,n 

{w  }  is  a  grid  which  becomes  dense  on  the  support  of  F  and 

lfn 

such  that  N'  >  N  implies  {w  }  C  {w  ,}  ; 

x  i  n  x )  n 

GO 

(iii)  either  G  ^(1)  <  00  or  G  *(1)  ■  “  ,  f  xdG(x)  <  00  and  gG  ^ 

0 

has  a  continuous  derivative  on  [0,1]  . 

Then 

r(x  )  <  lim  inf  r  (x  )  <  lim  sup  r  (x  )  <  r(x  ) 

\  o  I  —  no  —  r  n  o  —  '  o ' 

with  probability  one. 

Proof : 

To  show  lia  sup  r^(xQ)  .  The  right  hand  inequality  is  trivial  if 

r(x  )  ■  0  or  r(x+)  ■  “  ;  otherwise,  let  x,  >  x  satisfy  r(x, )  <  ® 

O  'O’  to  1 


and  let 
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a^(n) 

Nx(n) 


Let  w 


since 


Let  Y 

G*(x)  ■ 

where 

Hence, 


■  a.  be  the  index  of  the  largest  grid  point  w.  *  w  <  x,  .  Let 


J  J.n  ~  1 


■  and  ^(n)  ■  be  defined  by 


r  (x  )  -  —  . 
n  o  N2~^ 


rA\) '  F"(”“i) 


'•(\)  -  F»(\) 


"V1 


k  G"1F(w^ 


I  r(x  ) 


and  note  that 


*  * 
Wj+1  "  Wj 


J+l 

l 


xIh1_  du  <  w  _  w 

r(Xl)  dU  -  Wj+1  Wj 


j  +  1  <_  a.^  implies  w^+1  <_  x.^  and  r(x)  <_  r(x1)  for  x  <_  x1  .  Hence, 


r  (x)  <_- — r 
n  —  a^-l 


H\) '  M\) 


ilh  g°'lr“("j,(Vl  '  "i) 


G_1F(X1) 

r(xx) 


where  X^X^  . . . ,  Xq  is  an  ordered  sample  from  F  and 


P{Y*  <.x}  -  G(r(x1)y)  .  Let  G*(x)  -  G^v*)  *>  for  w*  I  x  <  w*+i 


*  * 


G  is  the  empirical  distribution  corresponding  to  Y^  Y2  ^  Y^  . 
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r(x)  1  a  1 

n  —  a^-l 


^  "'XHKvi  -  uj) 

v(°“(\))  *  v(<(\)) 


*  *  *  * 

Since  w._  <  x  while  w  t  x.  with  probability  1, 

—  o  a^  1 


lim  inf  g/w  \  -  G  ( w  \ 

”1  V  nl  V 


>  o 


with  probability  1.  (Note  G  (xq)  <  G  (x^)  since  G  is  increasing  on  its  support 

* 

an  interval.)  It  follows  from  Theorem  2.1  (letting  F  -  G  ),  that 


lim  sup  r  (x  )  <  lim  sup 
n  o 

n+*>  n-x» 


* , 

°n| 

(  *  \ 
w 

alJ 

1  -  Gn(”» 

v(<(\))  -  V1 

(\)) 

r  (x. ) 


with 


3  •  S  • 


probability  1,  since  $  *(y)  ^  *  $>  *(y)  ■  ,  x  uniformly  for  0  <_  y  <_  1 

G  G  rul; 


Letting  x^  I  xq  ,  we  see  that 


lim  sup  rn(xQ)  <  r(x+) 


n-+® 


with  probability  1. 

By  a  similar  argument,  we  have 

r(x  \  <  lim  inf  r  (x  ) 
Vo/—  no 


with  probability  1.  | 
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Remark  1: 

Note  that  the  key  to  the  proof  was  the  uniform  convergence  of 
3  S 

$  *( y)  ‘  4  *(y)  which  does  not  depend  upon  F  .  Hence,  a  similar  consistency 

G  G 

n 

proof  can  be  given  for  the  case  F  >  G  .  In  this  circumstance,  the  estimator 
becomes 


F  (w  )  -  F  (w  ) 


r  (x) 
n 


sup 


inf 


n  t^n 


n  s ,  n 


t-1 


t-1+l  Sli  l  gG-1F  (S  )(w  -  v  ) 

n  J  j+1.n  j,n 


Remark  2: 

For  the  grid  based  on  order  statistics,  the  estimator  r  (x)  is  not 

n 

consistent.  However,  as  Rao  (1966)  has  shown,  isotonizing  effectively  widens  the 

"windows"  so  that  r  is  strongly  consistent. 

n 

Remark  3:  Random  Window  Estimators 

There  are  at  least  two  ways  of  specifying  window  estimators.  One  can  specify 

the  length  of  the  windows  and  let  the  number  of  observations  in  a  window  be  random 

cr  one  can  specify  the  number  of  observations  in  a  window  and  thus  allow  the 

interval  length  to  be  random.  In  practice,  it  may  sometimes  be  more  convenient  to 

specify  the  number  of  observations  in  a  window  rather  than  the  window  length.  The 

MLE  for  r(t)  assuming  r  nondecreasing  is  the  isotonic  regression  of  a  random 

window  estimator  which  allows  precisely  one  observation  per  window.  Suppose  we 

specify  random  window  intervals  to  be  of  the  form  [X  ,X  \  for 

\  Un  ]  [  (i+l)n  j  / 

i  ■  0,1,  ...  and  some  8(0  <_  8  <  1)  .  Strong  consistency  of  random  window 
estimators  can  be  verified  either  by  modifying  the  proof  of  Theorem  2.1  for  this 
case  or  by  the  method  to  be  discussed  in  the  next  section,  since  now  the  basic 


estimator  is  consistent. 


4.  CONSISTENCY  OF  ISOTONIC  ESTIMATORS  WHEN  THE  BASIC  ESTIMATOR  IS  CONSISTENT 


I 


\ 
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Brunk  (1965)  Introduced  a  general  method  for  proving  consistency  of  isotonic 
estimators  when  the  basic  estimator  is  consistent.  We  generalize  his  results  some¬ 
what  in  order  to  treat  the  generalized  failure  rate  function.  A  fundamental 

difference  between  Brunk  estimators  and  our  estimators,  r  ,  defined  on 

n 

ft  (ft  C  ft)  is  that  we  extend  the  domain  to  ft  and  define  a  version  of  r  on 

n  n  n 

ft  .  We  then  wish  to  prove  that  this  version  converges  strongly  on  ft  (assuming 

ft  -  ft)  . 
n 

Let  (ft.A.y)  be  a  measure  space  and  let  A  be  a  set  of  A-measurable 
functions  on  ft  with  the  following  properties: 

(a)  *i  ,  A2  e  A  implies  max  (A^.A^  and  min  (A^.A^  e  A  , 

(b)  A  n  L2(u)  is  convex  and  closed  in  , 

(c)  A  is  a  complete  inner  product  space  with  inner  product 

(A.,A  )  *  f  A  (w)A  (w)du(w)  . 
ft 

For  any  £  c  L2  ,  there  exists  1  t  A  fl  Lj  such  that 


(4,1) 


/ 

ft 


a  - 


A)2du 


'-S 


u  - 


£)2dy 


/ 


u  - 


A;2dy 


for  all  i  £  A  (1  Lj  .  This  £  ■  P(£  |  AH  L2)  is  unique  [y]  .  (Cf.  Brunk 
(1965),  Theorem  2.1  and  Corollary  2.1.)  P(£  |  A  fl  L2)  is  the  projection  of  £  on 
A  n  l2  . 


Lemma  4.1: 

If  tor  some  e  A  ,  £  <_  A2  a.e.  [y]  ,  then  £  n-e>  [w]  •  ^  for 

some  A^  k  A  ,  £  >_  A^  a.e.  [y]  ,  then  £  >_  A^  a.e.  [y)  . 
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Proof : 

Let  £  <_  X^  a.e.  [p]  .  If  l  >  ^  on  a  set  of  positive  p  measure, 
consider  A  =  min  (Z,X^)  .  Since  A  is  A-measurable  and  min  (£,£)  <_  A  <_  £  ,  A 
is  square  integrable.  Because  of  Condition  (a),  A  £  A  ,  so  A  e  A  D  .  On  the 
set  where  £  <_  X^  ,  A  *  £  .  However  on  the  set  where  £  >  A^  ,  which  has  positive 
p-measure,  £  <_  A^  ■  A  <  £  .  Hence, 

(£  -  A)2  <  (£  -  £) 2 

on  ft  with  strict  inequality  on  a  set  of  positive  measure  and  therefore 

y  (£  -  A)2du  <  y  (£  -  £>2du 

ft  a 

which  contradicts  (3.1).  The  proof  of  the  second  part  of  the  lemma  is  similar.  || 
Note  that  Brunk's  (1965)  conditions  A^  <_  0  and  X^  >_  0  or  p(ft)  <  »  in  his 
Corollary  3.3  are  unnecessary. 

Lemma  A . 2  : 

If  for  some  A^  ,  A^  e  A  ,  A^  ^  £  <_  A^  a.e.  [p]  ,  then  there  exists  a 
version  of  £  for  which 

Xl  1  1  I  A2 

on  ft  .  [Note  that  £  is  defined  only  on  sets  of  positive  p-measure.] 

Proof: 

Let  £  be  any  version  of  P(£  |  A  D  L^)  on  ft  and  let 

l  -  min  (A ^  ,  max  (£,A^))  . 
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Since  X^  <_  l  <_  X2  a.e.  [ u ]  ,1*1  a.e.  [p]  .  Also,  X^  <  i  <  X^  on  ft  and 
*  >*■  —  /» 

£  e  A  because  of  Condition  (a).  Finally,  since  l  =  l  a.e.  [u]  ,  and  Z  is 

a 

A-measurable,  l  e  .  | | 

We  now  replace  Condition  (a)  by 

(A)  X^  e  A  for  all  r  e  T  (arbitrary  index  set)  implies 

sup  X  ,  inf  X  £  A  . 
teT  1  teT  T 

This  implies  that  every  function  $  on  ft  for  which  there  exists  X  £  A  with 
$  <_  X  has  a  smallest  majorant  $  e  A  ;  if  no  such  \  e  A  exists,  we  set  E  *  . 
Similarly,  if  there  exists  X  <_  <p  ,  X  e  A  ,  then  $  possesses  a  largest  minorant 
|  e  A  ;  if  no  such  X  e  A  exists,  we  set  .  From  Lemma  4.2,  we  have: 

Lemma  4.3: 

There  exists  a  version  of  l  for  which  _£  <_  l  <_  I  on  0  . 

Note  that  we  can  construct  such  a  version  on  the  basis  of  £_  and  I 

(cf.  proof  of  Lemma  4.2).  From  now  on,  we  shall  only  consider  those  versions  of 
l  that  are  indicated  in  Lemma  3.3.  Suppose  Condition  (A)  holds  and  (b)  is 
satisfied  for  a  sequence  of  measures  .  Then  we  have: 

Theorem  4,4: 

If  l  c  L„(y  )  and  l  and  l  are  uniformly  (strongly)  consistent 
n  2  n  _n  n  1  °  J 

estimates  of  the  same  X  e  A  ,  then  »  P(£^  |  A  ft  L£)  is  also  a  uniformly 
(strongly)  consistent  estimator  of  X  . 

Proof ; 

By  Lemma  4.1  and  4.2,  |£  —  X I  <  e  and  II  —  X i  <  e *  on  ft  implies 

1  ’  '_n  1  —  n  1  n  1  —  n  r 

I  £  -  X  I  <  max  (e  , e ' )  .  H 

1  n  1  —  n’  n  11 

~  * 

Theorem  4.4  can  be  used  to  prove  the  (strong)  consistency  of  r  (x)  ,  r  (x) 

n  n 
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and  r  (x)  when  the  basic  estimator 
n 


F_  n  _)  "  F  (W  ) 

a\  n  X « 1 1  n  n  X  %  n 

) - - « - -» - 

1  ‘  (wi+l,n  "  Wi,n>8G'  W 


is  (strongly)  consistent.  For  example,  if  f  is  a  uniformly  continuous  density 
function  and 


w.,.  -  v. 

i+1 , n  i ,  n 


0  <  a  <  1 


then  r^(x)  is  strongly  uniformly  consistent.  [Cf.  Nadarya  (1965),  Theorem  1.] 

To  utilize  Theorem  4.4,  we  need  to  verify  the  uniform  strong  consistency  of 

r  and  r  .  Since  it  is  not  obvious  that  this  condition  will  be  realized  in 
n  n 

general,  we  must  utilize  a  device  employed  by  Marshall  and  Proschan  (1965).  Let 

M 

F  ■  {F  I  F  <  G}  where  G  is  specified  and  F  *  (F  I  F  <  G  and  r(x)  <  Ml  where 

C  C  ”* 

r(x)  ■  f(x)/gG  F(x)  .  Let  r  be  the  isotonic  estimator  of  r  subject  to  F^ 

n  n 

in  F^  .  Assuming  rn(x)  uniformly  strongly  consistent,  we  can  show  that 

M  -M 

r  and  r  (x)  are  uniformly  strongly  consistent  in  the  same  way  that  the 
_ n  n 

Glivenko-Cantelli  lemma  proves  that  F  (x)  is  uniformly  strongly  consistent. 

n 

Furthermore,  r  converges  ir.  a  natural  way  to  r  as  M  -►  00  . 
n  n 
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